fix openapi spec import for specific urls #907

Udhay-Adithya · 2025-10-01T22:25:04Z

PR Description

The OpenAPI import feature was failing when trying to import specifications from URLs like https://catfact.ninja/docs?api-docs.json. The error "The fetched content does not look like a valid OpenAPI spec (JSON or YAML)" was shown even though the content was a valid OpenAPI 3.0 specification. This was caused by a bug in the openapi_spec package (v0.15.0) that cannot parse OpenAPI specs containing "security": [[]] (empty security arrays), which is valid according to the OpenAPI 3.0 specification.

Related Issues

Closes #

Checklist

I have gone through the contributing guide
I have updated my branch and synced it with project main branch before making this PR
I am using the latest Flutter stable branch (run flutter upgrade and verify)
I have run the tests (flutter test) and all tests are passing

Added/updated tests?

Yes
No, and this is why: please replace this line with details on why tests have not been included

OS on which you have developed and tested the feature?

Windows
macOS
Linux

Udhay-Adithya · 2025-10-01T22:25:46Z

Added a workaround in OpenApiImportService.tryParseSpec() that detects parsing failures and automatically removes problematic security fields containing empty arrays before retrying the parse operation. This is a temporary workaround until the upstream package is fixed.

animator · 2025-10-02T12:06:05Z

@Udhay-Adithya Can you add links to more real world openAPI spec URLs for which our parser works or doesn't works?

Udhay-Adithya · 2025-10-05T18:38:42Z

@Udhay-Adithya Can you add links to more real world openAPI spec URLs for which our parser works or doesn't works?

APIDash - Works without any fix
Cat Fact API - Works with this fix
DigitalOcean Droplet Metadata API - Works without any fix
GitHub v3 REST API - Doesn't Work
Swagger Petstore - Doesn't Work
RailwayStations REST API - Doesn't Work
UniProt REST API Server - Doesn't Work
VIT-AP VTOP API - Works without any fix

Udhay-Adithya · 2025-10-05T18:48:46Z

It’s not our parser that causes the issue. The failures come from the documents themselves and how the openapi_spec package (correctly) enforces OpenAPI shapes. Valid security fields work fine as per the package docs; the broken cases are due to invalid spec content.

Findings per document

cat_facts.json (also the Cat Facts URL)
- Problem: Top-level security is malformed: security: [[]]
- Why it fails: In OpenAPI 3.0, top-level security must be an array of SecurityRequirement objects (maps). Examples:
  - Valid: security: [] (no requirements) or security: [ { api_key: [] } ]
  - Invalid: security: [[]] (array of arrays)
- openapi_spec error: type 'List' is not a subtype of type 'Map<String, dynamic>'
- Conclusion: The document is invalid. This is not a general “security field” issue, just this malformed shape.
railway-stations.yaml
- Problem: Component parameter reference points to a Parameter missing required fields (e.g., 'in').
- Error: CheckedFromJsonException: Could not create Parameter. There is a problem with "in". Invalid union type "null"!
- The stack/message points at $ref: #/components/parameters/Authorization.
- Conclusion: Not related to security. The referenced Parameter definition is incomplete (missing in: header|query|path|cookie) or otherwise invalid.
travel.yaml
- Problem: Same class of failure as railway-stations.yaml, with a parameter ref like $ref: #/components/parameters/page.
- Error: CheckedFromJsonException... problem with "in" (Invalid union type "null").
- Note: components.securitySchemes is present here and is not the cause.
- Conclusion: Also a spec issue with parameter component definitions/references.
digitalocean.yaml
- Result: Parses successfully with openapi_spec.
- Note: No top-level security; nothing problematic here.
- Conclusion: Confirms the parser handles valid documents correctly.

Steps to reproduce failures from local files,

import 'dart:io';
import 'package:openapi_spec/openapi_spec.dart';

void main(List<String> args) async {

  // Pass file paths as args below.
  final paths = args.isNotEmpty
      ? args
      : <String>[
          './cat_facts.json',
          './railway-stations.yaml',
        ];

  for (final p in paths) {
    stdout.writeln('\n=== Parsing: $p ===');
    final f = File(p);
    if (!await f.exists()) {
      stdout.writeln('Skip: file not found');
      continue;
    }

    final content = await f.readAsString();

    try {
      final spec = OpenApi.fromString(source: content, format: null);
      stdout.writeln('SUCCESS: title="${spec.info.title}", version="${spec.info.version}"');
      stdout.writeln('Paths: ${spec.paths?.length ?? 0}');
    } catch (e, st) {
      final err = e.toString();
      stdout.writeln('FAIL: ${err.substring(0, err.length.clamp(0, 400))}...');
      // Stack Trace
      final stStr = st.toString();
      if (stStr.isNotEmpty) {
        stdout.writeln('Stack:\n$stStr');
      }
    }
  }
}

How to run

Create a new dart project, put the openapi spec file and this script there.
Add the depndency, dart pub add openapi_spec: ^0.15.0
Run:
- dart run path/to/this/file

Expected outcomes

cat_facts.json
- FAIL with an error like:
  - type 'List' is not a subtype of type 'Map<String, dynamic>'
- This is triggered by the invalid top-level security shape: security: [[]]
railway-stations.yaml
- FAIL with an error like:
  - CheckedFromJsonException: Could not create Parameter. There is a problem with "in". Invalid union type "null"!
- This points to a components/parameters reference missing required “in”.

animator · 2025-10-06T18:25:55Z

@Udhay-Adithya these comments are insightful. I will create a dev doc adding your comments and merge this PR. THank you.

Copilot

Pull Request Overview

This PR fixes OpenAPI spec import functionality for URLs containing specifications with empty security arrays, which previously failed due to a bug in the openapi_spec package. The solution implements a workaround that temporarily removes problematic security fields during parsing.

Adds a workaround for openapi_spec package parsing issues with empty security arrays
Implements preprocessing logic to detect and handle problematic security fields
Adds comprehensive test coverage for the new parsing scenarios

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
lib/dashbot/services/openapi_import_service.dart	Implements workaround logic with preprocessing methods to handle empty security arrays
test/dashbot/services/openapi_import_service_test.dart	Adds test cases covering the new parsing scenarios and edge cases

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-11T20:57:49Z

lib/dashbot/services/openapi_import_service.dart

+    } catch (e) {
+      // Try workaround for security field parsing issues
+      try {
+        final processedSource = _removeProblematicSecurityField(source);
+        if (processedSource != source) {
+          return OpenApi.fromString(source: processedSource, format: null);
+        }
+      } catch (_) {
+        // Workaround failed, fall through to return null
+      }
      return null;
    }


The workaround logic has a potential issue: if _removeProblematicSecurityField throws an exception, it's caught and ignored, but the method returns null instead of re-throwing the original parsing exception. This could mask the actual parsing error. Consider logging the original exception or providing more specific error handling.

Copilot · 2025-10-11T20:57:49Z

lib/dashbot/services/openapi_import_service.dart

+
+      return source;
+    } catch (e) {
+      throw FormatException('Failed to preprocess OpenAPI spec: $e');


The error message could be more specific about what preprocessing failed. Consider including details about the security field processing or JSON parsing failure to help with debugging.

Copilot · 2025-10-11T20:57:50Z

lib/dashbot/services/openapi_import_service.dart


+  /// Removes problematic security fields that cause parsing issues.
+  /// TODO: Remove this workaround once openapi_spec package fixes
+  /// the issue with security fields containing empty arrays.


The TODO comment should include a reference to track when this workaround can be removed, such as a GitHub issue number or version number of the openapi_spec package that will contain the fix.

Suggested change

/// the issue with security fields containing empty arrays.

/// the issue with security fields containing empty arrays.

/// See: https://github.com/omidyar/openapi_spec/issues/123 (update with actual issue number)

animator · 2025-10-11T21:06:10Z

@copilot Create a openAPI spec document in doc/dev_guide folder with all the above reported examples, issues and comments made by Udhay-Adithya in this PR.

fix: handle security fields with empty arrays

4652344

animator added the gsoc25 label Oct 2, 2025

animator requested a review from Copilot October 11, 2025 20:57

Copilot AI reviewed Oct 11, 2025

View reviewed changes

animator assigned Udhay-Adithya and Copilot Oct 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix openapi spec import for specific urls #907

fix openapi spec import for specific urls #907

Uh oh!

Udhay-Adithya commented Oct 1, 2025

Uh oh!

Udhay-Adithya commented Oct 1, 2025

Uh oh!

animator commented Oct 2, 2025

Uh oh!

Udhay-Adithya commented Oct 5, 2025

Uh oh!

Udhay-Adithya commented Oct 5, 2025 •

edited

Loading

Uh oh!

animator commented Oct 6, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 11, 2025

Uh oh!

Copilot AI Oct 11, 2025

Uh oh!

Copilot AI Oct 11, 2025

Uh oh!

animator commented Oct 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	/// the issue with security fields containing empty arrays.
	/// the issue with security fields containing empty arrays.
	/// See: https://github.com/omidyar/openapi_spec/issues/123 (update with actual issue number)

fix openapi spec import for specific urls #907

Are you sure you want to change the base?

fix openapi spec import for specific urls #907

Uh oh!

Conversation

Udhay-Adithya commented Oct 1, 2025

PR Description

Related Issues

Checklist

Added/updated tests?

OS on which you have developed and tested the feature?

Uh oh!

Udhay-Adithya commented Oct 1, 2025

Uh oh!

animator commented Oct 2, 2025

Uh oh!

Udhay-Adithya commented Oct 5, 2025

Uh oh!

Udhay-Adithya commented Oct 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Findings per document

How to run

Expected outcomes

Uh oh!

animator commented Oct 6, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 11, 2025

Choose a reason for hiding this comment

Uh oh!

animator commented Oct 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Udhay-Adithya commented Oct 5, 2025 •

edited

Loading